UIP Paper Reading

Posted on

Unprocessing Images for Learned Raw Denoising

文章贡献有两点：

  • Unprocess操作：转一个Clean图像到RAW。Process操作：逆Unprocess。
  • 添加dataset-dependent noise 噪声，生成与真实噪图相同的噪图。

image|690x409,75%
image|510x209

训练：

数据集：

  1. million images of the MIR Flickr extended dataset setting aside 5% of the dataset for validation and 5% for
    testing
  2. 将每一张来自Internet的数据集图像，这里使用的是来自MIR Flickr的图像。对其进行Unprocess，得到Raw Image，作者说该Unprocess pipeline是可逆的，可逆操作为Process。
  3. 在得到Raw Image后根据数据集的特征参数来添加shot and read噪声(Thus the values of λread and λshot can be calculated by the camera for a particular exposure and are usually stored as part of the metadata accompanying a raw image file.)。

  4. 在添加了噪声的RAW图像中进行去噪器的训练得到RAW的去噪结果，For both experiments, we minimize L1 loss between the output and ground-truth images.

    • 针对SRGB模型，our “sRGB” model the network output and synthetic ground-truth are both transformed to sRGB space before computing the loss，
    • 针对RAW模型， Our “Raw” model instead computes the loss directly between our network output and our raw synthetic ground-truth, without this processing. 直接RAW对应loss计算，不转为sRGB。

  5. 测试在DND上进行测试。

噪声，以及domain-aimed 噪声添加

RAW域的噪声分为两种噪声：1.是光子到达噪声，2.是sensor读取噪声。Sensor noise primarily comes from two sources: photon arrival statistics (“shot” noise) and imprecision in the readout circuitry (“read” noise)

两者结合，噪声图像可以看作：其中 $y$ 是观测后验。
image|470x67,75%

$\lambda{read} = g_d^2 \sigma_r^2$ , $\lambda{shot}=gd g_a$ 是由sensor’s 模拟，数字增益，以及固定读取方差 $\sigma_r^2$ 决定的。 $\lambda{read}$ $\lambda_{shot}$ 是可以通过相机的particular exposure and are usually stored as part of the metadata accompanying a raw image file.

作者计算了DND的所有 $log \lambda{read}$ $log\lambda{shot}$ ,下图蓝色所示，对其拟合得到红色线（加置信度）。
image|552x500,50%
image|664x217,75%
从其中抽样各个噪声，添加到RAW图中进行生成。

code of read and shot add->dataset 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
def random_noise_levels():
  """Generates random noise levels from a log-log linear distribution."""
  log_min_shot_noise = tf.log(0.0001)
  log_max_shot_noise = tf.log(0.012)
  log_shot_noise = tf.random_uniform((), log_min_shot_noise, log_max_shot_noise)
  shot_noise = tf.exp(log_shot_noise)

  line = lambda x: 2.18 * x + 1.20
  log_read_noise = line(log_shot_noise) + tf.random_normal((), stddev=0.26)
  read_noise = tf.exp(log_read_noise)
  return shot_noise, read_noise


def add_noise(image, shot_noise=0.01, read_noise=0.0005):
  """Adds random shot (proportional to image) and read (independent) noise."""
  variance = image * shot_noise + read_noise
  noise = tf.random_normal(tf.shape(image), stddev=tf.sqrt(variance))
  return image + noise

def create_example(image):
  """Creates training example of inputs and labels from `image`."""
  image.shape.assert_is_compatible_with([None, None, 3])
  image, metadata = unprocess.unprocess(image)
  shot_noise, read_noise = unprocess.random_noise_levels()
  noisy_img = unprocess.add_noise(image, shot_noise, read_noise)
  # Approximation of variance is calculated using noisy image (rather than clean
  # image), since that is what will be avaiable during evaluation.
  variance = shot_noise * noisy_img + read_noise

  inputs = {
      'noisy_img': noisy_img,
      'variance': variance,
  }
  inputs.update(metadata)
  labels = image
  return inputs, labels